In recent years, with an improvement of computer performance, there has been proposed that a gesture UI (User Interface) and the like be realized using a depth detected by a stereo camera that picks up images from different points of view by two cameras.
Since a search direction is a horizontal direction only in stereo matching performed during depth detection, the stereo matching is based on the premise of a state of no perpendicular misalignment between left and right cameras (a state in which epipolar lines of the left and right cameras match with each other and are in parallel). Therefore, it is necessary to correct (make a rectification to) a rotational misalignment and the like of the stereo camera in sub-pixel order by an image process.
The rectification needs to use parameters acquired by a calibration of each stereo camera and characteristic of the stereo camera. Examples of a calibration method include, for example, a method of picking up a chart pattern on which a plurality of feature points, among which a positional relationship is known, are printed a plurality of times while changing points of view (see, for example, NPL 1).
However, it is difficult for an inexpensive stereo camera for use as a consumer product or the like to mechanically ensure geometrical position accuracy between the left and right cameras, and the parameters of the stereo camera possibly change with an over-time misalignment thereof.
To address the problem, there has been proposed that a misalignment between left and right images that is possibly generated due to the over-time misalignment of the stereo camera be simply modeled and the misalignment between the left and right images be corrected in accordance with a resultant model (see, for example, PTL 1). The correction method described in PTL 1 can readily correct the misalignment since an angle of view, a resolution, and the like of the stereo camera are unnecessary.
In the present specification, the over-time misalignment of the stereo camera means a variation of the stereo camera from initial calibration time and examples of the over-time misalignment include a deformation due to a force applied to a housing from outside, and a deformation of a material with temperature.